The Shell gives you the ability to combine programs in many interesting ways. This is its primary strength.

Commands

date: prints today's date

echo: prints its arguments. You can chain this off of another program to print that program's output.

man: gives more information about a program

tldr: a program you can install that gives additional information atop man

cd: change directory

Note that the shell comes with auto-completion, so you can often complete paths faster by pressing <TAB>!

pwd: prints the current working directory

which: prints the directory set in $PATH associated with the given program

ls: lists the content of the current working directory, or given directory.

Consider installing and using eza for a more human-friendly ls.

cat <file>: Prints the contents of file

Consider installing and using bat over cat for syntax highlighting and scrolling.

sort <file>: Prints out the lines of file in sorted order

uniq <file>: Eliminates consecutive duplicate lines from file

head <file> and tail <file>: Respectively print the first and last few lines of file

grep <pattern> <file>: finds lines matching pattern in file.

pattern is a Regular Expression
You can also specify a directory instead of a file (or leave it off for .) and pass -r to recursively search all the files in a directory.

sed <arguments>: programmatically edit files.

See #Sed

find: recursively finds files given parameters

See #Find

awk: parses files

See #Awk

Directories

Absolute paths start with / Relative paths start from the current working directory.

There are also two “special” components that exist in every directory: . and ... . is “this directory”, and .. is “the parent directory”. So:

missing:~$ cd /
missing:/$ cd bin/../bin/../bin/././../bin/..
missing:/$

Consider installing and using zoxide to speed up your cding — z will remember the paths you frequently visit and let you access with less typing.

$PATH

When you type a command into the shell, it consults an environment variable named $PATH that lists all directories of programs it could execute.

missing:~$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
missing:~$ which echo
/bin/echo
missing:~$ /bin/echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

The : in the echoed path above are separators.

You can bypass $PATH by giving the full path to the program you wish to run.

Consider installing and using eza for a more human-friendly ls.

Grep

grep matches a given Regular Expression against files containing strings. See #Commands for an example of usage.

Consider installing and using ripgrep over grep for a faster and more human-friendly (but less portable) alternative. ripgrep will also recursively search the current working directory by default!

Sed

sed is a programmatic file editor. Give it a file and various arguments in its language, and it can make edits to files based on what you give it. For example,

missing:~$ sed -i 's/pattern/replacement/g' file

replaces all instances of pattern with replacement in file. The -i indicates that we want to edit the file directly. Without it, sed will simply print the modified output.

Find

find is a program that helps you recursively find files given certain conditions.

For example,

missing:~$ find ~/Downloads -type f -name "*.zip" -mtime +30

Finds ZIP files in the Downloads directory that are older than 30 days, and

missing:~$ find ~ -type f -size +100M -exec ls -lh {} \;

Finds files larger than 100M in your home directory and lists them.

The -exec option takes a command terminated with a ;.

You can also search by file contents, such as

missing:~$ find . -name "*.py" -exec grep -l "TODO" {} \;

which finds .py files with the string "TODO" in them.

Consider installing and using fd instead of find for a more human-friendly (but less portable!) experience.

Awk

awk is another that has its own programming language. It is intended for parsing files. Mostly useful for data files with consistent syntax like csv or json, and allows you to extract certain parts of lines.

For example,

missing:~$ awk '{print $2}' file

Prints the second whitespace-separated column of every line of file.

All Together, Now

Putting these tools together, we can do fancy things like:

missing:~$ ssh myserver 'journalctl -u sshd -b-1 | grep "Disconnected from"' \
  | sed -E 's/.*Disconnected from .* user (.*) [^ ]+ port.*/\1/' \
  | sort | uniq -c \
  | sort -nk1,1 | tail -n10 \
  | awk '{print $2}' | paste -sd,
postgres,mysql,oracle,dell,ubuntu,inspur,test,admin,user,root

This grabs SSH logs from a remote server (we’ll talk more about ssh in the next lecture), searches for disconnect messages, extracts the username from each such message, and prints the top 10 usernames comma-separated.

Bash

Pipes `|`

Pipes | let you plug the output of one program into another. Whatever would normally be printed to the terminal, is instead "piped" into whatever program you give it.

`>file`

Takes the output of a program and writes it to file instead of your terminal. >>file will append it instead of overwriting the file.

<file lets your read from file as a program's input instead of your keyboard.

tee will print outputs like cat, but will also write it to a file. For example, verbose cmd | tee verbose.log | grep CRITICAL will preserve the full verbose log to a file while keeping your terminal clean!

Conditionals

if: checks if the run program did not result in an error. then: if no error, then it will run the specified program. else: otherwise, run this.

The most common command to use as your if command is test, often abbreviated simply as [, which lets you evaluate conditions like “does a file exist” (test -f file / [ -f file ]) or “does a string equal another” ([ "$var" = "string" ]). In bash, there’s also [[ ]], which is a “safer” built-in version of test that has fewer odd behaviors around quoting.

Loops

while executes a command repeatedly as long as it does not result in an error.

while command1; do command2; command3; done for executes a command a number of times based on supplied variables.
for varname in a b c d; do command; done executes command four times, each time with $varname set to one of a, b, c, and d.
Instead of listing the items explicitly, you’ll often use “command substitution”, such as: for i in $(seq 1 10); do

In older code you’ll sometimes see literal backticks (like for i in `seq 1 10`; do) instead of $(), but you should strongly prefer the $() form as it can be nested.

Scripting

You will, of course, generally want to build complex programs in Shell Scripts .sh instead of writing them all directly in your terminal.

For example, here’s a script that will run a program in a loop until it fails, printing the output only of the failed run, while stressing your CPU in the background (useful to reproduce flaky tests for example):

#!/bin/bash
set -euo pipefail

# Start CPU stress in background
stress --cpu 8 &
STRESS_PID=$!

# Setup log file
LOGFILE="test_runs_$(date +%s).log"
echo "Logging to $LOGFILE"

# Run tests until one fails
RUN=1
while cargo test my_test > "$LOGFILE" 2>&1; do
    echo "Run $RUN passed"
    ((RUN++))
done

# Cleanup and report
kill $STRESS_PID
echo "Test failed on run $RUN"
echo "Last 20 lines of output:"
tail -n 20 "$LOGFILE"
echo "Full log: $LOGFILE"

Exercises

What does the -l flag to ls do? Run ls -l / and examine the output. What do the first 10 characters of each line mean? (Hint: man ls)
In the command find ~/Downloads -type f -name "*.zip" -mtime +30, the *.zip is a “glob”. What is a glob? Create a test directory with some files and experiment with patterns like ls *.txt, ls file?.txt, and ls {a,b,c}.txt. See Pattern Matching in the Bash manual.
What’s the difference between 'single quotes', "double quotes", and $'ANSI quotes'? Write a command that echoes a string containing a literal $, a !, and a newline character. See Quoting.
The shell has three standard streams: stdin (0), stdout (1), and stderr (2). Run ls /nonexistent /tmp and redirect stdout to one file and stderr to another. How would you redirect both to the same file? See Redirections.
$? holds the exit status of the last command (0 = success). && runs the next command only if the previous succeeded; || runs it only if the previous failed. Write a one-liner that creates /tmp/mydir only if it doesn’t already exist. See Exit Status.
Why does cd have to be built into the shell itself rather than a standalone program? (Hint: think about what a child process can and cannot affect in its parent.)
Write a script that takes a filename as an argument ($1) and checks whether the file exists using test -f or [ -f ... ]. It should print different messages depending on whether the file exists. See Bash Conditional Expressions.
Save the script from the previous exercise to a file (e.g., check.sh). Try running it with ./check.sh somefile. What happens? Now run chmod +x check.sh and try again. Why is this step necessary? (Hint: look at ls -l check.sh before and after the chmod.)
What happens if you add -x to the set flags in a script? Try it with a simple script and observe the output. See The Set Builtin.
Write a command that copies a file to a backup with today’s date in the filename (e.g., notes.txt → notes_2026-01-12.txt). (Hint: $(date +%Y-%m-%d)). See Command Substitution.
Modify the flaky test script from the lecture to accept the test command as an argument instead of hard-coding cargo test my_test. (Hint: $1 or $@). See Special Parameters.
Use pipes to find the 5 most common file extensions in your home directory. (Hint: combine find, grep or sed or awk, sort, uniq -c, and head.)
xargs converts lines from stdin into command arguments. Use find and xargs together (not find -exec) to find all .sh files in a directory and count the lines in each with wc -l. Bonus: make it handle filenames with spaces. (Hint: -print0 and -0). See man xargs.
Use curl to fetch the HTML of the course website (https://missing.csail.mit.edu/) and pipe it to grep to count how many lectures are listed. (Hint: look for a pattern that appears once per lecture; use curl -s to silence the progress output.)
jq is a powerful tool for processing JSON data. Fetch the sample data at https://microsoftedge.github.io/Demos/json-dummy-data/64KB.json with curl and use jq to extract just the names of people whose version is greater than 6. (Hint: pipe to jq . first to see the structure; then try jq '.[] | select(...) | .name')
awk can filter lines based on column values and manipulate output. For example, awk '$3 ~ /pattern/ {$4=""; print}' prints only lines where the third column matches pattern, while omitting the fourth column. Write an awk command that prints only lines where the second column is greater than 100, and swaps the first and third columns. Test with: printf 'a 50 x\nb 150 y\nc 200 z\n'
Dissect the SSH log pipeline from the lecture: what does each step do? Then build something similar to find your most-used shell commands from ~/.bash_history (or ~/.zsh_history).

Sources:

https://missing.csail.mit.edu/2026/course-shell/

The Shell